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DETAILED ACTION 



Response to Amendment 



The amendments filed on 09/03/2003 have been entered. Claims 1, 4-9, 11, 14-17, and 21-28 
have been amended. Claims 2, 12, 19-20 have been canceled. 



1. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

1. Claims 1, 3-1 1, 13-18, 21-28 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Bradski et al. U.S. Pat. No. 6,363,160 (hereinafter Bradski) in view of Zabih et al. U.S. Patent 
No. 6,181,817 (hereinafter Zabih), Kenet et al. U.S. Patent No. 5,016,173 (hereinafter Kenet) and 
Darrell et al. U.S. Patent No. 6,188,777 (hereinafter Darrell). 

2. Claim 1: 

(1) Bradski teaches an automated calibration system (figure 1; column 3, lines 34-55) to 
track a selected object through a series of fi-ames of data (column 7, lines 55-65), comprising: 

A display device (e.g., figure 1 1) to display at least one image fi-ame received from an 
image input device, wherein the image frame includes a calibration rectangle (column 4, lines 2- 
60; column 15, lines 13-58; figures 4, lOA and lOB); 



Claim Rejections - 35 USC §103 
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An image selection device to select, via the calibration rectangle, the selected object in 
the at least one image frame (column 3, lines 59-67, column 4, lines 1-10, and column 7, lines 
33-54; column 15, lines 13-58); 

An image source device (video source 1 102 of figure 1 1) to provide a hue saturation 
value (HSV) data array of pixels forming the at least one image frame (column 4, lines 2-57; 
column 13, lines 45-52; column 15, lines 13-58; figures 3A and 3B); and 

A processing device (figures 1-9; column 13, Unes 55-67; column 14, lines 1-59) to 
determine analysis data for pixels (column 4, lines 2-57; figures 4, 3A and 3B) within the 
calibration rectangle (column 5, Unes 52-67; column 6, lines 1-44; column 15, lines 13-58; 
figures 4, lOA and lOB), based on the HSV data array (column 4, lines 2-57; figures 3 A and 3B), 
and determine test analysis data (e.g., window sizing parameters) for a set of adjacent test 
windows (e.g., search windows or boxes or windows of FIG. 14A displaying images; column 5, 
lines 52-67; column 6, lines 1-44; column 15, lines 13-58), each of the adjacent test windows 
(e.g., the moving search windows) having a same shape as the calibration rectangle (e.g., the 
entire video frame or a calculation region in a form of rectangle or a search window; column 7, 
lines 5-55; column 15, lines 13-58), wherein tracking data, to track the selected object (e.g., the 
selected gestures), is selected from one of the calibration rectangle (e.g., the calculation region in 
a form of rectangle) and the adjacent test windows (e.g., the search windows) having a highest 
tracking probability (i.e., the tracking data such as window location parameters are 
determined/adjusted and a search window having the largest comiected region of a probability 
distribution and the greatest probability density is selected, column 6, lines 45-67; column 7, 
lines 55-67; column 8, Unes 1-9; column 15, lines 13-58) and each of the adjacent test windows 
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share at least one pixel with the calibration rectangle (e.g., column 7, lines 5-55; column 15, lines 
13-58). 

(2) However, Bradski is silent on the claim limitation of "lowest sum of a hue standard 
deviation and a saturation standard deviation". 

(3) Darrell in view of Zabih and Kenet teaches selecting an object based on the "lowest 
sum of a hue standard deviation and a saturation standard deviation." (e.g., Darrell column 10, 
lines 44-65; Zabih column 2, lines 20-37; Kenet column 20, lines 10-30; Kenet column 16, lines 
1-20). Darrell teaches selecting a tracking object based on the optimum ranges and the highest 
probability density of a search region by collecting the variance data of joint probability 
distributions for the observed color data while Zabih teaches selecting a best match result (with 
the optimum ranges and the highest probability density) for an object based on the color joint 
histograms of the candidate images and based on the calculation of the standard deviations 
(variance) for each color in HSV color space while Kenet teaches finding a search region based 
on the optimum ranges/criteria by optimization and joint histograms. 

(4) It would have been obvious to one of ordinary skill in the art to have incorporated the 
Darell/Zabih/Kenet's selection means with the optimum ranges/criteria into the Bradski's 
invention to select a tracking object based on the lowest sum of a hue standard deviation and a 
saturation standard deviation. Bradski teaches performing pattern recognition based on Gaussian 
and Chi-square distributions to determine a best match for the tracked object and therefore 
suggests selecting a tracking object based on the lowest sum of a hue standard deviation and a 
saturation standard deviation because such a selection of optimum ranges yields the best match 
tracking object. 
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(5) One having the ordinary skill in the art would have been motivated to do this because 
it would have provided a routine experimentation of the optimum ranges and criteria for object 
selection to find the best match tracking object. 



Claim 3: 

The claim 3 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of a digital camera. However, Bradski further discloses the claimed limitation 
of the digital camera (Bradski column 4, lines 1-57). 

Claim 4: 

The claim 4 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of a single device. However, Bradski further discloses the claimed limitation 
of the single device (figure 1). 

Claim 5: 

The claim 5 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of the processing device calculating a mean hue and a standard deviation of a 
hue of the pixels representing the selected object. However, Bradski further discloses the claimed 
limitation of the processing device (figures 1- 9) calculating a mean hue and a standard deviation 
of a hue of the pixels representing the selected object (figures 3 A, 3B and 4; column 4, lines 2- 
57, column 10, lines 1-16, column 5, lines 31-63, and column 6, lines 22-44; column 12, lines 1- 
67). 

Claim 6: 
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The claim 6 encompasses the same scope of invention as that of claim 5 except additional 
claimed limitation of the selected object not being tracked if the mean hue or the standard 
deviation of the hue is less than predetermined levels. However, Bradski further discloses the 
claimed limitation of the selected object not being tracked if the mean hue or the standard 
deviation of the hue is less than predetermined levels (It is apparent that a Gaussian or Chi- 
Square hue probability distribution of Bradski involves the mean hue or the standard deviation of 
the hue and search for the area with the greatest probability density involves the motion-tracking 
processing of not tracking a region where the mean hue or the standard deviation of the hue 
being less than predetermined levels; figures 3 A, 3B and 4; column 4, lines 2-57, column 10, 
lines 1-16, column 5, lines 1-63, and column 6, lines 22-44; column 8, lines 10-38; column 12, 
lines 1-67). 

Claim 7: 

The claim 7 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of the analysis module calculating a mean saturation and a standard deviation 
of a saturation of the pixels representing the selected object. However, Bradski further discloses 
the claimed limitation of the processing device (e.g., figures 1-9) calculating a mean saturation 
and a standard deviation of a saturation of the pixels (e.g., a mean saturation and a standard 
deviation of a saturation of the pixels is an inherent property of a Gaussian or Chi-Square 
probability density) representing the selected object (figures 3A, 3B and 4; column 4, lines 2-57, 
column 10, lines 1-16, column 5, lines 31-63, and column 6, lines 22-44; column 8, Hnes 10-38; 
column 12, lines 1-67). 

Claim 8: 



# 
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The claim 8 encompasses the same scope of invention as that of claim 7 except additional 
claimed limitation of the colored object not being tracked if the mean saturation or the standard 
deviation of the saturation is less than predetermined level However, Bradski further discloses 
the claimed limitation of the selected object not being tracked if the mean saturation or the 
standard deviation of the saturation is less than predetermined level (figures 3 A, 3B and 4; 
column 4, lines 2-57, column 10, lines 1-16, column 5, lines 1-63, and column 6, lines 22-44; 
column 8, lines 10-38; column 10, lines 35-67; column 12, lines 1-67). 

3. Claim 9: 

The claim 9 encompasses the sanie scope of invention as that of claim 1 in a method 
form. The claim is rejected for the same reason as set forth in claim 1 . 



The claim 10 encompasses the same scope of invention as that of claim 9 except 
additional claimed limitation of converting a pixel data array for the at least one image frame 
from a red-green-blue color space (RGB) data array to the HSV data array. However, Bradski 
further discloses the claimed limitation of converting a pixel data array for the at least one image 
frame from a red-green-blue color space (RGB) data array to the HSV data array (column 4, 
lines 1-57). 



The claim 1 1 encompasses the same scope of invention as that of claim 28 except 
additional claimed limitation of applying the pixel data from an entire frame to the pixel- 
classification look-up map wherein if the amount of the pixels associated with the selected object 



Claim 10: 



Claim 11: 
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is greater than a predetermined amount, the calibration method restarts. However, Bradski 
further discloses the claimed limitation of applying the pixel data from an entire frame to the 
pixel-classification look-up map (column 4, lines 45-60) wherein if the amount of the pixels 
associated with the selected object is greater than a predetermined amount, the calibration 
method restarts (figures 3 A, 3B and 4; column 4, lines 1-61; column 6, lines 5-57; column 10, 
lines 35-67). 



Claim 13: 

The claim 13 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of thresholding the HSV data array of pixels and disregarding pixel 
data for each of the pixels having a product of a saturation coordinate and a value coordinate 
below a predetermined threshold amount. However, Bradski further discloses the claimed 
limitation of thresholding the HSV data array of pixels and disregarding pixel data for each of 
the pixels having a product of a saturation coordinate and a value coordinate below a 
predetermined threshold amount (From the Bradski' s teaching that "Hue values are only 
accumulated if their corresponding S and V values are above respective S and V thresholds"..., it 
can be seen that pixel data with the product of S and V values less than a threshold would be 
excluded from accumulation to the Hue values. See figures 3A, 3B and 4; column 4, lines 1-57; 
column 5, lines 1-61; column 10, lines 35-67; column 13, lines 55-67; column 14, lines 1-59). 
Claim 14: 

The claim 14 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of calculating a mean hue and a standard deviation of a hue of the 
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pixels in the selected object. However, Bradski further discloses the claimed limitation of 
calculating a mean hue and a standard deviation of a hue of the pixels (The mean hue and 
standard deviation of a hue values are inherently associated with hue probability density) in the 
selected object (column 4, lines 2-57, column 10, Hnes 1-16, column 5, lines 31-63, and column 
6, lines 22-44; column 8, lines 10-38; column 10, lines 35-67; column 12, lines 1-67). 
Claim 15: 

The claim 15 encompasses the same scope of invention as that of claim 14 except 
additional claimed limitation of restarting the calibration method if the mean hue or the standard 
deviation of the hue is less than predetermined level. However, Bradski further discloses the 
claimed limitation of restarting the calibration method (restarting searching) if the mean hue or 
the standard deviation of the hue is less than predetermined level (figures 3 A, 3B and 4; column 

4, lines 2-57, column 10, lines 1-16, column 5, lines 31-63, and column 6, lines 1-44; column 8, 
lines 10-38; column 10, lines 35-67; column 12, lines 1-67). 

Claim 16: 

The claim 16 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of calculating a mean saturation and a standard deviation of a 
saturation of the pixels in the selected object. However, Bradski further discloses the claimed 
limitation of calculating a mean saturation and a standard deviation of a saturation of the pixels 
in the selected object (figures 3A, 3B and 4; column 4, lines 2-57, column 10, Hnes 1-16, column 

5, lines 31-63, and column 6, lines 22-44; column 12, lines 1-67). 

Claim 17: 
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The claim 17 encompasses the same scope of invention as that of claim 16 except 
additional claimed limitation of restarting the calibration method if one of the mean saturation 
and the standard deviation of the saturation is less than predetermined level. However, Bradski 
further discloses the claimed limitation of restarting the calibration method if one of the mean 
saturation and the standard deviation of the saturation is less than predetermined level (figures 
3 A and 3B; column 4, lines 2-57, column 10, lines 1-16, column 5, lines 31-63, and column 6, 
lines 22-44; column 12, lines 1-67). 

Claim 18: 

The claim 18 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of allowing the user to select the selected object. However, Bradski 
further discloses the claimed limitation of allowing the user to select the selected object (column 
4, Unes 1-57; column 15, Unes 1-67). 

Claim 21: 

The claim 21 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of a thresholding module to disregard pixel data for each of the 
pixels having a product of a saturation coordinate and a value coordinate below a predetermined 
threshold amount. However, Bradski flirther discloses the claimed limitation of a thresholding 
module to disregard pixel data for each of the pixels having a product of a saturation coordinate 
and a value coordinate below a predetermined threshold amount (figures 3 A, 3B, 4-5; column 4, 
lines 1-57; column 13, lines 55-67; column 14, lines 1-59). 

Claim 22: 
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The claim 22 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of the calibration rectangle being smaller than the at least one 
image frame. However, Bradski further discloses the claimed limitation of the calibration 
rectangle being smaller than the at least one image frame (figures 4, lOA-lOB and 14 A; column 
6, lines 1-67; column 15, lines 1-67). 



The claim 23 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of each of the adjacent test windows having a same size as the 
calibration rectangle. However, Bradski further discloses the claimed limitation of each of the 
adjacent test windows having a same size as the calibration rectangle (figures 4, lOA-lOB and 
14A; column 6, lines 1-67; figures 11-14, column 15, lines 1-67). 



The claim 24 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of each of the adjacent test windows having at least one pixel 
overlapping with the calibration rectangle. However, Bradski further discloses the claimed 
limitation of each of the adjacent test windows having at least one pixel overlapping with the 
calibration rectangle (figures 4, lOA-lOB and 14A; column 6, lines 1-67; figures 1 1-14, column 
15, lines 1-67). 



The claim 25 encompasses the same scope of invention as that of claim 9 except 
additional claimed limitation of the calibration rectangle being smaller than the at least one 
image frame. However, Bradski further discloses the claimed limitation of the calibration 



Claim 23: 



Claim 24: 



Claim 25: 
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rectangle being smaller than the at least one image frame (figures 4, lOA-lOB, and 14A; column 
6, lines 1-67; column 15, Unes 1-67). 
Claim 26: 

The claim 26 encompasses the same scope of invention as that of claim 9 except 
additional claimed limitation of each of the adjacem test windows having a same size as the 
calibration rectangle. However, Bradski further discloses the claimed limitation of each of the 
adjacent test windows having a same size as the calibration rectangle (figures 4, lOA, lOB and 
14A; column 6, lines 1-67; figures 11-14, column 15, lines 1-67). 

Claim 27: 

The claim 27 encompasses the same scope of invention as that of claim 9 except 
additional claimed limitation of each of the adjacent test windows having at least one pixel 
overlapping with the calibration rectangle. However, Bradski fiirther discloses the claimed 
limitation of each of the adjacent test windows (the moving search windows) having at least one 
pixel overlapping with the calibration rectangle (figures 4, lOA-lOB; column 6, lines 1-67; 
figures 11-14, column 15, lines 1-67). 
Claim 28: 

The claim 28 encompasses the same scope of invention as that of claim 9 except 
additional claimed limitation of creating a pixel-classification look-up map for the HSV data 
array of pixels, and the pixel classification map classifies the pixels belonging to the selected 
object based on a hue and a saturation of the pixels. However, Bradski further discloses the 
claimed limitation of creating a pixel-classification look-up map for the HSV data array of 
pixels, and the pixel classification map classifies the pixels belonging to the selected object 
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based on a hue and a saturation of the pixels (figures 4, lOA and lOB; column 4, lines 2-57; 
column 6, lines 5-35), 



Conclusion 

4. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jin-Cheng Wang whose telephone number is (703) 605-1213. 
The examiner can normally be reached on 8:00 AM - 4:30 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mike Razavi can be reached on (703) 305-4713. The fax phone numbers for the 
organization where this application or proceeding is assigned are (703) 308-6606 for regular 
communications and (703) 308-6606 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 395-3900, 

jew 

September 28, 2003 

MICHAEL RAZAVI 
SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 




